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SYSTEM AND METHOD FOR 
SIMULTANEOUS MEDIA PLAYOUT 



BACKGROUND OF THE INVENTION 
5 1. Field of the Invention 

This invention generally relates to multicast or unicast 
media communications and, more particularly, to a system and 
method for creating simultaneous media playout at multiple 
heterogeneous clients. 

10 2. Description of the Related Art 

In a streaming media system, a server typically streams 
media data to multiple clients. Each client has its own capability, 
which may be different than other clients. For example, each client 
may have different amount of buffering space available. The server 

15 streams the same media stream to all the clients in a multicast 
session. The goal is for all these clients with different buffering 
capabilities, is to achieve a simultaneous playback of the media 
stream. One application of a multicast session is a home-network 
environment where a single audio server streams a song to multiple 

20 client devices, perhaps in different rooms of a home, so that the 
clients achieve a simultaneous playout of a song. 

Because each client has different buffering capabilities, 
the server needs to adapt its streaming whenever a new client joins 
the session or leaves the session. When a media packet is 

25 transmitted from the server to the clients, the packet may have a 
variable delay before reaching the clients. This variation in the end- 
to-end delay is called jitter. As noted by Schulzrinne et al., "RTP: A 
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transport protocol for real-time applications", IETF, Jan. 2000, Sender 
and Receiver Report (SR, RR) packets of Real-time Transport Control 
Protocol (RTCP) part of Real-time Transport Protocol (RTP) have a field 
interarrival jitter that captures the mean deviation of the difference in 
5 packet spacing at the client compared to the server for a pair of 

packets. To compensate for the jitter, a streaming media client often 
buffers a certain number of packets, and then plays them out at 
scheduled playout times. A typical streaming media system often 
transports packets using RTP on top of User Datagram Protocol 

10 (UDP). UDP provides an unreliable service where transmitted packets 
can get lost, arrive out of order, or get duplicated. The client side 
buffering of data in a streaming media system helps to alleviate the 
out-of-order packet delivery by rearranging the buffered packets. The 
client can also discard the duplicate packets. The buffering is also 

15 helpful if there is any disruption in the streaming during the session, 
causing interruption in stream reception for a short time. 

The amount of buffering done at the client side in a 
streaming media system is based mainly on two factors: the available 
client buffer size, and the acceptable delay which the user can tolerate 

20 before the media actually starts playing, after the time of the request. 
The type of media encoding used may also impose a certain amount of 
delay before a client can actually start to decode the received packets. 
This is typically the case for video encoded using any of the popular 
video encoding standards (e.g. MPEG1, MPEG2, MPEG4, H.263(+)), 

25 where the frames are encoded independently (Intra = I frames) or by 
referring to other frames (Inter = P,B frames). The delay may also be 
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dependent upon whether the media is being played out is a live 
Stream or an on-demand archived stream. 

With respect to the first factor, the available client buffer 
size, each client can have a different buffer size. In the case where 
5 each client is playing the same media stream in a session, it is 
assumed that packets are application data units and are 
independently decodable. This is true for majority of the audio 
encoding standards. The goal is to achieve a simultaneous playout of 
the media stream at each client. Another problem is the case where 
10 clients can join or leave the session midway. Client heterogeneity and 
dynamic session membership are major problems to be addressed in 
multicasting. 

Adaptive playout delay adjustment and synchronization of 
streams have been proposed to address the above-mentioned 

15 problems. Assuming a media stream consisting of talkspurts 
interspersed with silence, adaptive playout algorithms have been 
proposed by Ramjee et al, "Adaptive playout mechanisms for 
packetizing audio applications in wide-area networks", Proceedings of 
IEEE INFOCOM, pp. 680-688, 1994. These algorithms estimate the 

20 mean and variation in end-to-end delay to adjust the starting time for 
playout of each talkspurt. However, this work does not address the 
issue of multiple heterogeneous clients having different buffering 
capabilities. It is also more targeted towards an interactive 
conferencing type of systems. Such a solution does not address an 

25 on-demand archived media (especially audio) distribution system, that 
has a continuous media stream without separate talkspurts. Neither 
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does this method address server side adaptation based on different 
client buffering capabilities. 

The other class of related work addresses the problem of 
synchronization between different media streams in a presentation 
5 session. For example, video and audio streams that need to be 

synchronously presented in a multimedia session. A start-up protocol 
to initiate synchronized playback of multiple media streams is 
proposed by Biersack et al ? "Synchronized delivery and playout of 
distributed multimedia steams", Multimedia Systems, Vol. 7, No. 1, 

10 pp. 70-90, January 1999. A scheme that allows audio and video 

stream synchronization using a local conference bus, is also proposed 
by Kouvelas et aL, "Lip Synchronization for use over the Internet", 
Proceedings of IEEE Globecom, Nov. 1996. These systems address a 
set of multiple streams transmitted from one or more servers to a 

15 single client, with the focus on achieving a synchronized playback of 
these multiple streams at that client. 

Yuang et aL, "Intelligent video smoother for multimedia 
communications", IEEE Journal of Selected Areas in 
Communications, Vol. 15, No. 2, pp. 136-146, Feb. 1997, describes 

20 an intelligent neural network based video smoother that compensates 
for jitter and smoothes the playout of video frames. However, these 
solutions do not address the issue of achieving simultaneous 
playback of the same media stream at multiple heterogeneous clients 
having different buffering capabilities. 

25 Further, although prior art systems describe client side 

buffering, none of known solutions appear to handle simultaneous 
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media playout at multiple clients, where each client has a different 
buffer capacity. Methods designed to achieve isochronous streams 
appearing simultaneously at output ports cannot handle 
simultaneous playout when the buffer capacity of each client is 
5 different. 

It would be advantageous if media could be played out at 
several clients simultaneously, even if the clients had different 
buffering capacities. 

It would be advantageous if simultaneous playout could 
10 be maintained despite disruptions in the media stream. 

It would be advantageous if simultaneous playout could 
be maintained despite changes in client membership during a session. 

SUMMARY OF THE INVENTION 

15 The present invention avoids the above-mentioned 

problems associated with clients having different buffering capacities 
by permitting simultaneous media playout strategies for multiple 
heterogeneous clients using server side adaptation. There are three 
adaptation phase problems that are solved by the invention. The 

20 adaptive phases are: when a client joins the session; when a client 
leaves the session; and, after streaming disruptions. 

Accordingly, a method has been provided for achieving 
simultaneous media playout in a network including a server and a 
plurality of clients. The method comprises: from a server, supplying a 

25 media stream to clients at a first bitrate (Rl); determining the network 
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delivery requirement; and, in response to the network delivery- 
requirements, modifying the supply of the media stream. 

Determining the network delivery requirements includes 
determining either the buffering capacities of the clients, or the media 
streaming disruptions. To determine the buffering capacities of the 
clients, a first minimum client buffering capacity (CI) is determined 
by polling the clients for their respective buffering capacities, and 
selecting the first minimum buffering capacity (CI) to be equal to the 
client with the smallest buffering capacity* 

The method further comprises: following the supplying of 
media stream at a first bitrate (Rl), changing clients in the network; 
determining the new minimum client buffering capacity (Cnew); and, 
in response to the new minimum buffering capacity (Cnew), modifying 
the supply of the media stream. The supply of the media stream is 
modified by temporarily pausing the supply of the media stream at 
the first bitrate (Rl) if the new minimum buffering capacity (Cnew) is 
less than the first minimum buffering capacity (CI). Alternately, the 
media stream bitrate is temporarily increased if the new minimum 
buffering capacity (Cnew) is greater than the first minimum buffering 
capacity (CI), or if it has been determined that the supply of the 
media stream has been disrupted. 

Additional details of the above-described simultaneous 
media playout method, and a system for achieving simultaneous 
media playout in a network are described below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic block diagram of the present 
invention system for achieving simultaneous media playout in a 
network. 

5 Figs. 2 through 4 illustrate three examples, where a new 

client having a lower buffering capability joins the session midway (at 
time m), thus lowering the buffer capacity. 

Figs. 5a through 5c illustrate additional examples of the 
present invention adaptation process. 
10 Figs. 6 through 8 illustrate a scenario where a client 

leaves the session midway (at time m), thus increasing the buffering 
capacity. 

Figs. 9a and 9b are flowcharts illustrating the present 
invention method for achieving simultaneous media playout in a 
15 network including a server and a plurality of clients. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 is a schematic block diagram of the present 
invention system for achieving simultaneous media playout in a 

20 network. The system 100 comprises a server 102 having a network 
connection port to supply a media stream at a first bitrate (Rl) on a 
network 104. The server 102 determines network delivery 
requirements and, in response, modifies the supply of the media 
stream. At least one client has a network connection port to receive 

25 the media stream. Shown are client one (106), client two (108), and 
client M (1 10). However, the invention is not limited to any particular 
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number of clients. The clients 106-1 10 transmit their buffering 
capacities to the server 102. 

The server 102 determines the network delivery 
requirements in response to determining the buffering capacities of 
5 the clients. Further, the server determines network delivery 
requirements in response to determining media streaming 
disruptions. 

With respect to determining the buffering capacity of the 
clients 106-110, the server 102 determines the first minimum client 

10 buffering capacity (CI). The server 102 polls the clients 106-1 10 for 
their respective buffering capacities and each client 106-110 
transmits their buffering capacity to the server 102, in response to the 
poll. The server 102 determines the first minimum buffering capacity 
(CI) to be equal to the client with the smallest buffering capacity. 

15 Each client schedules the playout to be at a time equal to 

a first time interval (tl) plus the minimum buffering capacity divided 
by the first bitrate (Cl/Rl). Prior to supplying a media stream at the 
first bitrate (Rl), the server 102 communicates the first minimum 
buffering capacity (CI) to the clients. Each client receives the value tl 

20 as a timestamp of the first buffered frame in the media stream. As 
part of the normal playout procedure, each client 106-1 10 reorders 
out-of-order media packets, handles any lost media packets, and 
schedules the playout of the media packets at a uniform rate in 
response to the media type and the first bitrate (Rl) because the 

25 media stream includes media packets with a timestamp. 
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One major problem addressed by the present invention is 
that of adding, or removing clients from the network. For example, 
after supplying the media stream at a first bitrate Rl to client one 
(106) and client two (108), an additional client, client M{1 10) is added 
5 to the network 104. The server 102 determines the new minimum 
client buffering capacity (Cnew), in response to the addition of the new 
client 110 and communicates the new minimum buffering capacity 
(Cnew) to the clients 106-110. Then, the server 102 modifies the 
supply of media stream in response to the new minimum buffering 

1 0 capacity (Cnew) . 

The addition or removal of clients from the network can 
either raise or lower the new minimum buffering capacity. If the 
buffering capacity is lowered, the server 102 modifies the supply of 
the media stream by temporarily pausing the supply of the media 

15 stream at the first bitrate (Rl). If the buffering capacity is increased, 
the server 102 temporarily increases the media stream bitrate. Prior 
to modifying the supply of the media stream, the server 102 
communicates the new minimum buffering capacity (Cnew) to the 
clients 106-110. 

20 More specifically, the server 102 temporarily pauses the 

supply of the media stream if the new minimum buffering capacity 
(Cnew) is less than the first minimum buffering capacity (CI). The 
server 102 temporarily pauses the supply of the media stream by 
pausing for a time equal to the first minimum buffering capacity 

25 minus the new minimum buffering capacity, divided by the first 
bitrate ((CI - Cnew)/Rl). Typically, the server 102 communicates 
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media stream modifications to the clients using real-time- streaming 
protocol (RTSP). However, the present invention is not limited to any 
particular type of protocol. There are many pre-exiting protocols that 
can be modified for use with the system 100. 
5 The server 102 temporarily increases the media stream 

bitrate if either the new minimum buffering capacity (Cnew) is greater 
than the first minimum buffering capacity (CI), or the server 102 
determines that the supply of the media stream has been disrupted. 
The server 102 temporarily increases the media stream bitrate if the 

10 new minimum buffering capacity is greater than the first minimum 
buffering capacity by polling the clients and itself to determine the 
minimum bitrate (Rmin). That is, the server 102 determines the 
maximum rate at which each client can receive a media stream, and 
the rate at which it can transmit. The device with the smallest 

15 bandwidth is the gating device, and the minimum bitrate (Rmin) is set 
to be equal to the smallest bandwidth. Note, that the minimum 
bitrate (Rmin) is greater than the first bitrate (Rl). Then, the server 
102 supplies the media stream at the minimum bitrate (Rmin) for a 
time equal to the new minimum buffering capacity minus the first 

20 minimum buffering capacity, divided by the minimum bitrate minus 
the first bitrate ((Cnew - CI) /(Rmin - Rl). 

To ease the above-mentioned calculations, the present 
invention uses the concept of a current_buffer_level and a 
target_buffer_level. The targetjDuffer_level is the same as the first 

25 minimum buffering capacity. Typically, the server 102 supplies media 
stream data as media packets. The server 102 first determines the 
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current buffer level (Cc), which is the number of media packets 
buffered at each client 106-1 10. If the new minimum buffering 
capacity is less than the current buffer level, the server temporarily 
pauses the supply of media stream for a time equal to the current 
5 buffer level minus the new minimum buffering capacity, divided by 
the first bitrate ((Cc - Cnew)/Rl). This calculation leads to the same 
results as the equation ((CI - Cnew)/Rl) mentioned above. 

If the new minimum buffering capacity is greater than the 
current buffer level, the server 102 polls the clients 106-1 10 and itself 

10 to determine the minimum bitrate (Rmin), as mentioned above, and 
supplies the media stream at the minimum bitrate (Rmin) for a time 
equal to the new minimum buffering capacity minus the current 
buffer level, divided by the minimum bitrate minus the first bitrate 
((Cnew - Cc) / (Rmin - Rl). The equation leads to the same results as 

15 equation ((Cnew -CI)/ (Rmin - Rl) mentioned above. 

The server 102 also temporarily increases the media 
stream bitrate if it has been determined that the media stream has 
been disrupted. The server 102 polls the clients 106-110 and itself to 
determine the minimum bitrate (Rmin), and supplies the media 

20 stream at the minimum bitrate (Rmin) for a time equal to new 

minimum buffering capacity minus the current buffering capacity, 
divided by the minimum bitrate minus the first bitrate ((Cnew - 
Cc)/(Rmin-Rl). 

In some aspects of the invention, the server 102 includes 

25 a tracker 1 10 to maintain a current-buffer-level measurement of the 
number of media packets being supplied by the server and a target- 
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buffer- level measurement of the new minimum client buffer. The 
server 102 modifies the supply of the media stream in response to the 
current-buffer-level and target-buffer-level measurements maintained 
by the tracker 110, using the equations mentioned above. 

Below, some examples are given of the above-mentioned 
system 100 in operation, and additional details to understand the 
problems solved with the use of the present invention. 

Adaptation phase when a client joins the session on the network 

Assume that the media is encoded, although the same 
analysis holds true for uncompressed media data, at a bitrate of Rl 
bits per second (bps). Frame-based media encoding is also assumed. 
Various audio coding standards (e.g. G.723, G.728, G.729, GSM, 
MP3, etc.) fall under this category. Also assume that there are M 
heterogeneous clients having buffer sizes {C x ,C 2 C M } bits, currently 

receiving the media stream. Every time a new client joins to receive 
the current media stream, the server queries its buffering capabilities. 
Assuming the client and server to be using Real-time Streaming 
Protocol (RTSP), the server could query the client buffering capability 
using a transaction similar to one shown below. 

Getting Client Buffer Size: 

Server -> Client : 

GET_PARAMETER rtsp: //myhomenw. com/f ilez/songl .mp3 

RTSP/1.0 

CSeq: 1089 

Content-type : text /parameters 
Session: 8438 
Content-length : 18 
CRLF 

buffer size bits 
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CRLF 

Client -» Server : 
RTSP/1.0 200 OK 
5 CSeq: 1089 

Content-length : 24 
Content-type : text /parameters 
CRLF 

buf f er_sizeJoits : 64000 
10 CRLF 



The server keeps track of the buffering capabilities of 
each client. Let C 1 bits denote the minimum buffer size of the 
currently participating clients, i.e. CI =Min(C l9 C 29 .... 9 C M ). It is assumed 

15 that each media packet streamed has a sequence number and a 
timestamp field (each RTP packet has a sequence number and a 
time stamp field). The sequence number can be used at the client side 
to detect duplicate, out-of-order and missing packets. The timestamp 
can be used to keep track of packet jitter and also to determine the 

20 play out scheduling of packets. Every time the buffering capacity 

changes as a result of a new client joining the media stream, with the 
buffering capability being equal to the least of the currently existing 
clients, the server communicates the new minimum buffering capacity 
(Cnew) value to all the clients. Assuming the server and client to be 

25 using RTSP, this can be done using a transaction similar to one 
shown below. 

Setting Client's Buffering Level: 

30 Server -» Client : 

SET_PARAMETER rtsp : //myhomenw . com/ file z/songl . mp3 RTSP/1.0 
CSeq: 1094 
Content-length: 18 
Content-type: text /parameters 
35 CRLF 

buf f ering__size: 384 00 
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CRLF 

Client ^Server : 
RTSP/1.0 200 OK 
5 CSeq: 1094 

Content-length: 7 
Content-type : text /parameters 
CRLF 
38400 
10 CRLF 



It is assumed that the server has also communicated the 
media stream first bitrate Rl. This will typically be done as a part of 
the Setup process, or could be done using the SETJPARAMETER 
15 method as above (assuming the server and clients to be using RTSP). 
The media stream itself may already carry the bitrate information (e.g. 
MP1, MP2, MP3, G.723.1). 

Every time a new client joins the session, after going 
through the setup and capabilities exchange process, the new client : 
20 • Buffers and re-orders the incoming packets, 

• Schedules the playout of the first buffered frame with timestamp = 

CI 

tl , at time (rl + — ) , 
Rl 

• Schedules the playout of the next buffered frames in order at the 
uniform rate based on the type of media encoding and the first 

25 bitrate Rl. 

Every time a new client joins, the server goes through the 
steps of setup and capabilities exchange. If the new client changes 
(lowers) the buffering capacity, assuming C 1 and Cnew refer to the old 
30 and the new minimum client buffer space, the server: 
• Communicates the Cnew value to all the clients. 
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• Pauses the media transmission for time (— — ^ new \ then 

Rl 

restarts the media delivery at the usual rate of 721 . If the server and 
client are using RTSP, then this pause in the media transmission is 
equivalent to a request from client as shown below : 

Pause rtsp: / /myhomenw. com/ f ilez/ songl .mp3 RTSP/1 . 0 
CSeq: 1145 
Range; npt=48 
Session: 8438 

where in the above example, the pause occurs at time 48 seconds 

and will last for (— — ^ new \ seconds. 

721 

During the adaptation period (pause period) of the server: 

• If a new client joins, thus lowering the current buffering capacity, 
the server can just augment the pause (increase it) based on the 
value of Cnew and communicate the Cnew value to the clients. 
Denoting the old Cnew value as CI, and the newer value as Cnew 
((Cnewjnewer). The total pause time duration, including the 
already paused time, will be (~~~ ' 

• If a client leaves, thus increasing the current buffering capacity, 
the server will complete the calculated pause before reacting to this 
buffering capacity change. The adaptation at the end of the pause 
period will be done according to the procedure described in the 
next section. 

Figs. 2 through 4 illustrate three examples, where a new 
client having a lower buffering capability joins the session midway (at 
time m), thus lowering the buffer capacity. The first example (Fig. 2) 
shows packet transmission at the server, packet reception at the 
client, and the packet piayout schedule at the client in the case when 
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there is no interarrival packet jitter, no out-of-order packet delivery, 
and no packet loss. The second example (Fig. 3) illustrates the 
situation of Fig. 2 with packet jitter and out-of-order packet delivery. 
The third example (Fig. 4) includes packet jitter, out-of-order packet 
5 delivery and packet loss. As can be seen from the figures, buffering 
can absorb the packet jitter and out of order packets. A packet loss 
could be handled using some receiver-based method. This can be a 
simple repetition of the last packet, a silent frame insertion, or some 
complex interpolation of data surrounding the missing data. 

10 

Adaptation phase when a client leaves the session 

Every time a client leaves the session (TEARDOWN 
message received, if using RTSP), the server computes new value of 
the minimum buffering capacity Cnew. If this value is higher than 

15 the previous value (CI), the server computes minimum of the 

maximum available reception bandwidth amongst all the clients. This 
is achieved by querying the maximum client bandwidth (bitrate) using 
GET_PARAMETER method. Based on the type of connection, this 
bandwidth (bitrate) may vary over time, so the server uses the most 

20 recent estimate of the client bandwidth (bitrate). Assuming the server 
and client using RTSP, this estimate can also be obtained from the 
client in its Bandwidth header field {section 12.6 of the real time 
streaming protocol (RTSP)}. Let 

Rclient^Min (RcIientl 9 RcIient2 9 .... 9 RcIientM)be the minimum of the 

25 maximum available client bandwidths and assume that R min > Rl . Let 
Kerver be ^ maximum rate at which the server can stream media 

data. It is assumed further that the media stream being transmitted 
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is already encoded and is available completely (as in case of an on- 
demand archived stream). Then the server can stream at a higher 

rate of Rmin = Min(Rclient,R server ) bps, for time T h = ^ new — — seconds 

R min- Rl 

and then revert back to the transmission rate of Rl bps. If the server 
5 and client are using RTSP, then this change in speed of transmission 
is equivalent to the requests from the client as shown below : 

Play rtsp: //myhomenw. com/f ilez/songl .mp3 RTSP/1.0 
CSeq: 1150 
10 Range: npt=104-120 
Speed: 1.2 
Session: 8438 

Play rtsp: //myhomenw. com/f ilez/songl ,mp3 RTSP/1 . 0 
15 CSeq: 1150 

Range: npt=120- 
Speed: 1.0 
Session: 8438 

. * * , J? min . . , Cnew - CI _ , 

20 Assuming that = 1.2, and : = 16 seconds, 

Rl R mm- Rl 

the server transmission change occurs at time 104 seconds. The 
server also communicates Cnew to all the clients (which being higher 
than previous Cnew (now called CI) can be used by the clients to get 
ready to receive media data at a higher rate). 
25 Figs. 5a through 5c illustrate additional examples of the 

present invention adaptation process. During the adaptation period 
(increased rate transmission time period) of the server, if a newer 
client joins at time t, thus lowering the buffering capacity: 

• Case 1: If (Cnew < (R min- Rl)t + CI) (Fig. 5a and 5b), the server 

30 immediately pauses the transmission for time 

(CI + (Rmin- Rl)t- Cnew) where f . g ^ ^ duration ^ 
v Rl 
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media was transmitted at rate R min . After this pause the server 
reverts back to the original transmission rate of Rl . 

• Case 2: If (Cnew > (Rmin-RY)t + CI) (Fig. 5c), the server calculates 

a newer value of Rmin, and streams at this increased rate for 

time ^ new — ^ — (i? min Rl)t ^ p^ er ^ s increased rate 
(J? min- Rl 

transmission time period the server reverts back to the original 
transmission rate of j?l . 

• Case 3: If (Cnew = (R min- Rl)t + CI) , server reverts the 

transmission to the rate Rl . 

The implementation of the above three cases can be 
simplified by assuming a hypothetical buffer at the server side. Two 
variables: current_buffer_level and targetj3ufferjevel can be used to 
keep track of the fill level and the target level of the hypothetical 
buffer. If a newer client joins, thus lowering the buffering capacity, 
the target_buffer_level is set to the Cnew. Then, the three cases above 
can be implemented as follows. 

• Case 1: If (target_buffer_level < current Joufferjevel), the 
transmission pauses for time 

(cur rent_buf f er_level - 1 arget„buf f er_level) ^ back tQ 

Rl 

the transmission at the rate of Rl . 

• Case 2: If (target_buffer_level > current_buffer_level), a new Rmin 
is found, where Rmin = Min(Rclient,R server ) , and the media stream is 

supplied at this increased rate for time 

(target_buffer.level-current,buffer_level ) ^ ^ media 

Rmin- Rl 

stream supply reverts back to rate Rl . 
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• Case 3: If (target jDufferJevel = current J}uffer_level), the media 
stream supply reverts back to the rate Rl . 

• If a client leaves, thus increasing the buffering capacity, the 
server does not react to this change until the end of the current 

5 increased speed interval. The server acts on this new change 

(with another possible increased speed transmission interval) at 
the end of the current interval. 

When a client leaves the session, thus increasing the 
buffering capacity, the server may choose to do no adaptation 

10 (increased rate transmission). This maybe because some clients do 
not support increased bandwidth reception, or because the server 
cannot handle the increased speed transmission adaptation phase for 
some reason. In this case, the system will work with a sub-optimal 
buffering behavior, which may result in some missed playout 

15 schedules. Essentially the system will continue to behave as if the 
lower buffering capacity client is still active. 

Figs. 6 through 8 illustrate a scenario where a client 
leaves the session midway (at time m), thus increasing the buffering 
capacity. The first case (Fig. 6) shows packet transmission at the 

20 server, packet reception at the client, and the packet playout schedule 
at the client in the case when there is no interarrival packet jitter, no 
out-of-order packet delivery, and no packet loss. The second case 
(Fig. 7) shows the same in the presence of the packet jitter and out-of- 
order packet delivery The third case (Fig. 8) includes packet jitter, 

25 out-of-order packet delivery, and packet loss. 
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Adaptation phase after streaming disruptions 

During the transmission session, the network load or 
competing traffic can result in congestion, and the session throughput 
from the server to clients can fall for a short duration. In this case 

5 after the congestion is over, the server can go through an adaptation 
phase similar to the adaptation phase when a client leaves the 
session, as mentioned above. As before, the period of high rate 
transmission can be calculated through the two variables: 
current_buffer_level and target Jauffer Jevel, to keep track of the fill 

10 level and the target level of the hypothetical buffer at the server side. 
The target_buffer Jevel is equal to the minimum client buffer size and 
the current Jniffer Jevel is determined by keeping track of the data 
streamed from the server and played out at the clients. In this case, 
because of the stream disruption, (targetjouffer Jevel > 

15 current Jsuffer Jevel), the server can find a new Rmin, where 

Rmm = Min(Rclient,R server ) , and stream at this increased rate for time 
(target_buf f er^level - current Jouf f er_leve l) ^ then reverts 

Rmin-Rl 

back to rateitt . If a new client joins during this adaptation phase, 
thus lowering the buffering capacity, targetjouffer Jevel is set to the 
20 newer minimum buffering capacity Cnew, and the 3 cases as 
described in the previous section are checked and acted on 
accordingly. This adaptation phase can also be carried out every time 
the difference (target_buffer Jevel - current Jauffer Jevel) exceeds some 
threshold. 

25 Figs. 9a and 9b are flowcharts illustrating the present 

invention method for achieving simultaneous media playout in a 
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network including a server and a plurality of clients. Although the 
method is depicted as a sequence of numbered steps for clarity, no 
order should be inferred from the numbering unless explicitly stated. 
The method begins at Step 900. Step 902, from a server, supplies a 
5 media stream to clients at a first bitrate (Rl). Step 904 determines 
the network delivery requirement. Step 906, in response to the 
network delivery requirements, modifies the supply of the media 
stream. 

Determining the network delivery requirements in Step 
10 904 includes determining the buffering capacities of the clients, or 
determining media streaming disruptions. When determining the 
buffering capacities of the clients, the first minimum client buffering 
capacity (CI) is determined. Determining the first minimum client 
buffering capacity (CI) in Step 904 includes substeps. Step 904a 
15 polls the clients for their respective buffering capacities. Step 904b 
determines which client has the smallest buffering capacity. Step 
904c selects the first minimum buffering capacity (CI) to be equal to 
the client with the smallest buffering capacity. 

A further step, Step 908, at each client, schedules the 
20 playout to be at a time equal to a first time interval (tl) plus the 

minimum buffering capacity divided by the first bitrate (Cl/Rl). Step 
901, prior to supplying a media stream at the first bitrate (Rl), 
communicates the first minimum buffering capacity (CI) to the 
clients. 

25 Following the supplying of media stream at a first bitrate 

(Rl) in Step 902, Step 905a changes clients in the network. Step 
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905b determines the new minimum client buffering capacity (Cnew). 
Step 905c communicates the new minimum buffering capacity (Cnew) 
to the clients. Step 906, in response to the new minimum buffering 
capacity (Cnew), modifies the supply of the media stream. Modifying 

5 the supply of the media stream in Step 906 includes temporarily 
pausing the supply of the media stream at the first bitrate (Rl), or 
temporarily increasing the media stream bitrate. 

Temporarily pausing the supply of the media stream at 
the first bitrate (Rl) in Step 906 includes temporarily pausing the 

10 supply of the media stream if the new minimum buffering capacity 
(Cnew) is less than the first minimum buffering capacity (CI). More 
specifically, Step 906 includes pausing for a time equal to the first 
minimum buffering capacity minus the new minimum buffering 
capacity, divided by the first bitrate ((CI - Cnew)/Rl). 

15 Determining the new minimum buffering capacity (Cnew) 

in Step 905b includes substeps. Step 905b 1 determines if the new 
minimum buffering capacity (Cnew) is greater than the first minimum 
buffering capacity (CI). Step 905b2 determines if the supply of the 
media stream has been disrupted. Modifying the supply of the media 

20 stream in Step 906 then includes temporarily increasing the media 
stream bitrate. 

Step 905d polls the clients and the server to determine 
the minimum bitrate (Rmin). Then, modifying the supply of the media 
stream in Step 906 includes supplying the media stream at the 

25 minimum bitrate (Rmin) for a time equal to the new minimum 
buffering capacity minus the first minimum buffering capacity, 
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divided by the minimum bitrate minus the first bitrate ((Cnew - 
Cl)/(Rmin-Rl). 

In some aspects of the invention, Step 905e, at the server, 
maintains a current_buffer_level measurement to track the number of 
media packets supplied by the server. Step 905f, at the server, 
maintains a target_buffer_level measurement to track the new 
minimum client buffering capacity (Cnew). Modifying the supply of 
the media stream in Step 906 includes modifying the supply of the 
media stream in response to the current_buffer_level and 
target_buffer_level measurements. 

Supplying a media stream from a server to clients at a 
first bitrate (Rl) in Step 902 includes supplying data as media 
packets with a timestamp. Step 905g determines the current buffer 
level (Cc), which is the number of media packets buffered at each 
client. Step 905h determines if the new minimum buffering capacity 
is less than the current buffer level. Modifying the supply of the 
media stream in Step 906 includes temporarily pausing the supply of 
media stream for a time equal to the current buffer level minus the 
new minimum buffering capacity, divided by the first bitrate ((Cc - 
Cnew)/Rl). 

Step 905d still polls the clients and the server to 
determine the minimum bitrate (Rmin). Modifying the supply of the 
media stream in Step 906 includes supplying the media stream at the 
minimum bitrate (Rmin) for a time equal to the new minimum 
buffering capacity minus the current buffer level, divided by the 
minimum bitrate minus the first bitrate ((Cnew - Cc)/(Rmin - Rl). 



sia!059 

-24- 

Determining the current_buffer_level in Step 905g 
includes determining that the media stream has been disrupted. 
Then, modifying the supply of the media stream in Step 906 includes 
supplying the media stream at the minimum bitrate (Rmin) for a time 

5 equal to new minimum buffering capacity minus the 

current_buffer_level, divided by the minimum bitrate minus the first 
bitrate ((Cnew - Cc)/(Rmin - Rl). 

Communicating the media stream modifications to the 
clients from the server in Steps (and Substeps of) 904 and 905 

10 typically includes using real- time-streaming protocol (RTSP). 

Scheduling the playout to be at a time equal to a first 
time interval (tl) plus the minimum buffering capacity divided by the 
first bitrate (Cl/Rl) in Step 908 includes substeps. Step 908a, at 
each client, reorders out-of-order media packets. Step 908b, at each 

15 client, handles any lost media packets. Step 908c, at each client, 
schedules the playout of the media packets at a uniform rate in 
response to the media type and the first bitrate (Rl). 

A system and method have been provided for adapting the 
distribution of media for simultaneous playout with heterogeneous 

20 clients. Some examples have been given of controlling the supply of 
media stream through the tracking of client buffering capacities. 
However, other variations and embodiments of the present invention 
will occur to those skilled in the art. 

25 
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